A headline reads: "Dale has the most break-ins of any town in the region." It is true. Dale has 3,600 reported break-ins a year. It also has 3 million people, and the town at the top of any sensible safety ranking might be a village of 15,000 where 60 break-ins are a worrying rate. The headline was accurate and misleading, and the difference is a single division.
Data journalism is the craft of finding and telling stories with numbers, and of not being fooled by them. It sits between reporting and statistics: you need a reporter's question, an analyst's care with the data, and a designer's honesty. Most of the work is not clever. It is checking definitions, dividing, comparing like with like, and showing what you did.
All towns, series and tables in this piece are invented and labelled as such. Real cases are named only where the record is clear.
Data stories start with a question, not a file. In July 1967 Detroit had one of the worst riots in US history; the Detroit Free Press asked its young reporter Philip Meyer to test the popular theory that rioters were uneducated newcomers from the South. His survey of residents found that the theory did not fit the evidence. A paper that had a theory now had a number to hold it against. Meyer went on to write Precision Journalism (1973), the founding text of the field, arguing that reporters should use the methods of social science.
Data work can also hold algorithms to account. In 2016 ProPublica reporters obtained risk scores used in courts in Broward County, Florida, matched them to later arrests, and reported that the tool wrongly labelled Black defendants who did not reoffend as high risk about twice as often as white ones. The software's maker disputed the conclusion, arguing that scores of a given level meant the same risk of reoffending for each group. Later academic work showed why both sides could be right: when groups have different underlying rates, no score can satisfy every reasonable definition of fairness at once. That is a data story at its best: a finding, a rebuttal and a clearer public question, not a verdict.
The usual stages are the same everywhere: question → find the data (government records, freedom-of-information requests, scraping what is public, leaks) → clean it → analyse → check with reporting (call the people behind the dots) → show it. The last check matters most: a pattern in a spreadsheet is a lead, and the story comes from reporting on why.
Raw counts mostly measure size. The biggest places have the most crimes, cases, deaths and complaints because they have the most people, so a ranking by count is often a ranking by population. The fix is a rate: events divided by the people (or households, or kilometres, or hours) exposed to them, then scaled to a readable unit such as per 100,000.
Per-capita is not magic. It assumes everyone in the denominator is equally at risk (a town full of tourists or commuters is not), and a good story asks who is really in the base. But it is the first fix. Two related traps: a "50% increase" on a tiny base can be three cases, and a rise of 2 percentage points (from 10% to 12%) is a 20% relative rise. Say which one you mean.
Invented towns. The same six places rank in nearly the opposite order.
Without JavaScript: by raw count Dale (3,600) is first and Elm (60) last; per 100,000 Elm (400) is first and Dale (120) last.
Rates fix one problem and create another. A town of 1,000 people with one extra event has a rate that jumps by 100 per 100,000. The statistician Andrew Gelman has used a famous example: in US maps of kidney-cancer death rates by county, the lowest-rate counties and the highest-rate counties were both mostly small, rural ones. The reason is not something in the water or the air. Small samples are noisy, and noise produces extremes in both directions.
The press has to be careful here. "Village has the nation's highest cancer rate" is exactly the story a rate-ranked list will keep producing, and the right response is to ask how many events it rests on. Good data teams use a minimum count, pool several years, or draw uncertainty bands. The simulation below gives every town the same true rate. Run it a few times.
Simulated with a seeded random-number generator; events follow a Poisson distribution. The dashed line is the true rate. Axis is capped at 400.
Without JavaScript: when every town has the same true rate, the highest and lowest observed rates both belong to towns far smaller than the typical one, because small counts swing.
Darrell Huff's How to Lie with Statistics (1954) and Edward Tufte's The Visual Display of Quantitative Information (1983) both ask whether a graphic's size on the page matches the size of the change in the numbers. Alberto Cairo's How Charts Lie (2019) is the modern guide. Four of the commonest tricks:
Invented monthly complaint counts. Try Jan–Mar with the axis from 410, then Mar–Oct, then the full year with the axis from 0.
Without JavaScript: across the whole year complaints fell about 0.9% (430 to 426). January to March they rose 5.1%; March to October they fell 7.5%. A truncated axis makes every one of those look dramatic.
The honest version shows the full period, starts the axis where the reader expects, says what the unit is, and puts the surprising number in the headline only when the chart shows it fairly. Newsroom style guides increasingly add a rule: if a chart was cut to fit, say where and why.
Before analysing, interview the dataset. Who collected it, to count what, with what definition? Did the definition change? Many real "trends" are changes in counting: a new reporting system, a change in what counts as a case, a shift from paper to digital. Case counts of any disease depend on how many people are tested; crime figures depend on whether victims report; a country's statistics office may be under pressure, so cross-check with an independent source (the World Bank, UN agencies or a university) and say where they differ. Africa Check's work on official crime and inequality statistics is built on this habit.
Then look at the raw rows: values of 999 or -1 (missing, coded as a number), a column mixing units, duplicates, and typing slips. These are the numbers that do the damage, because one stray 999 can drag an average far from the truth.
Invented emergency response times in minutes. Press each fix and watch the average.
Without JavaScript: the raw average is 133.0 minutes; after treating 999 as missing, converting 0.2 hours to 12 minutes and dropping the duplicate, it is 11.3 minutes.
The most famous warning is Berkeley. In the autumn of 1973 about 44% of men and 35% of women who applied to the graduate programmes were admitted, which looked like clear bias. Peter Bickel, Eugene Hammel and J. W. O'Connell (Science, 1975) split the numbers by department and found that, in most departments, women's admission rates were as high or higher. Women had applied more to departments that rejected more people. The combined figure was real, and what it meant needed the split.
This is Simpson's paradox: a trend in the whole can reverse inside every group when the groups are different sizes. It is not a trick of arithmetic; it is a warning that a cause lives in the structure behind the totals. The choice of what to split by cannot be settled by the data alone. A reporter decides it by knowing how the process works, and says so. The same care applies to causation: two lines that move together (ice cream and drownings) are usually both driven by something else, here summer. See Statistics (34/05) for the general case.
Invented numbers in the shape of the Berkeley case. Dept A admits most applicants; Dept B admits few, and most women applied there.
Without JavaScript: in Dept A men are admitted at 80% and women at 90%; in Dept B men at 20% and women at 30%. Combined, men are admitted at 70% and women at 40%.
Data journalism earns trust by being checkable. Good practice now includes a methodology note (where the data came from, how it was cleaned, what was excluded), the code or a spreadsheet where possible, and an honest statement of uncertainty. For a poll that means the margin of error; for a model it means a range, not a point. The best newsrooms also publish corrections to data stories as readily as to words. And they ask the human question the numbers cannot answer: if a chart shows that one neighbourhood has the highest rate of something, what is it like to live there, and does the person who lives there recognise it?
A number is not a story until it is compared with something. Divide by the right base, distrust the extremes from small places, draw the chart a sceptic would accept, audit the data before using it, and look inside the total. Then call the people the numbers describe.
Try this: find a "top ten" list in the news and look up the population of each place; does the order survive dividing?
Sources. Meyer, Precision Journalism and the Detroit Free Press 1967 riot survey (the paper won a Pulitzer Prize in 1968 for its riot coverage); ProPublica, "Machine Bias" (23 May 2016) and the maker's published response; Kleinberg, Mullainathan and Raghavan (2016) on incompatible fairness criteria; Gelman and Nolan on the kidney-cancer map and Wainer, "The Most Dangerous Equation" (2007); Bickel, Hammel and O'Connell, Science 187 (1975); Huff, Tufte, Cairo. All towns, series and tables here are invented; every number was computed.